Anomaly Detection: How to Artificially Increase Your F1-Score with a Biased Evaluation Protocol

Anomaly detection is a widely explored domain in machine learning. Many models are proposed the literature, and compared through different metrics measured on various datasets. The most popular used to compare performances F1-score, AUC AVPR. In this paper, we show that F1-score AVPR highly sensitive contamination rate. One consequence it possible artificially increase their values by modifying train-test split procedure. This leads misleading comparisons between algorithms especially when evaluation protocol not well detailed. Moreover, cannot be datasets as they do reflect intrinsic difficulty of modeling such data. Based these observations, claim should for anomaly detection. We recommend generic procedure unsupervised detection, including use other AUC, which more robust arbitrary choices protocol.